Peranan ANALISIS 


FAKTOR dalam 


Analisis Butir 


Wahyu Widhiarso | Fakultas Psikologi UGM 


\ 
fat) Peranan 
bn Analisis 
i AN Faktor dalam 
“teş Seleksi Butir 
| dea 


Tiga Analisis Utama dalam Psikometrika 


TEORI TES KLASIK ANALISIS FAKTOR TEORI RESPONS BUTIR 


Chapter 14 


INTRODUCTION 
TO 


Chapter 15 


LINDA CROCKER 
JAMES ALGINA 


UNIT IV Item Analysis in Test Development 


Item Analysis 311 

Item Difficulty, Mean, and Variance 311 

Item Discrimination 313 

Item Reliability and Validity Indices 320 
Conducting an Item Analysis Study 321 

Item Analysis for Criterion-Referenced Tests | 329 
Summary 335 

Exercises 336 


Introduction to Item Response Theory 339 


Basic Concepts of Item Response Theory 340 
The Normal Ogive 346 

Relating Item Response Theory to Classical Test 
Theory 350 

Logistic Models 352 

Estimation of Parameters 355 
Selection of a Model 361 
Applications of Item Response Theory 
Summary o 371 

Exercises 371 


362 


Teori Tes Klasik 


e Biasa dipakai sebagai landasan untuk melakukan analisis butir 
e Memiliki beberapa properti psikometris (daya beda butir, tingkat 
kesulitan butir, efektivitas distraktor, reliabilitas pengukuran dsb) 


Teori Tes Modern 


e Biasa dipakai sebagai landasan untuk melakukan analisis butir 
e Memiliki beberapa properti psikometris (daya beda butir, tingkat 
kesulitan butir, fungsi informasi butir-tes) 


e Scale Refinement 
e Purification 
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e Item Calibration 
e Scaling - Linking - Equating 
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Muncullah Analisis Faktor 
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e Scaling - Linking - Equating 


Ketiganya memiliki 
Properti yang mirip 


Teori Tes Klasik 


me e Daya Diskriminasi 
Analisis Faktor e Tingkat Kesulitan Butir 
Teori Tes Modern 


Pendekatan 1 


Analisis 


Analisis Butir 
Falacor 


Tes Klasik 


Step 1: Item Generation 


Step 2: Questionnaire Administration 


Step 3: Initial Item Reduction 


+ 
Step 4: Confirmatory Factor Analysis 


Step 5: Convergent/ Discriminant Validity 


Step 6: Replication 


Hinkin, 1998 


Apa alasannya? 


Analisis butir dulu baru 
pengujian validitas struktural 


ITEM 
GENERATION & 
SELECTION 
PHASE 


SCALE 
REFINEMENT 
PHASE 


SCALE 
VALIDATION 
PHASE 


Item 
Generation 
and 
Selection 


Pilot Test 
Sample 
(n = 203) 


Calibration 
Sample 
(n = 624) 


Validation 
Sample 
(n = 348) 


Qualitative Inquiry 
Stage 
e Content and face validity 
assessment 


Validation Stage 

* Replication of confirmatory 
factor analysis 

e Unidimensionality and 
reliability assessment 

e Convergent and discriminant 
validity assessment 

* Nomological validity 
assessment 


[Phase 1] Clarification of concept 


Development [Phase 2] Development of initial items 


of Scale 
165 items, 4 factors 
[Phase 3] Content validity of initial items 
(Expert panel review) 

92 items, 4 factors 

fres smst 
= 75 items, 4 factors 
[== 37 items, 4 factors 

Evaluation ee | 
of Scale 30 items, 3 factors 


Discriminative validity analysis 


Criterion validity analysis 


i 


Reliability analysis 


Yoon, 2021 


Lieraure |_| Demiin |__| Senised 
Review (LR) | a siya 
and Strategies (9 people) 


Exploratory 
Phase 


Focus group-2: Focus group-1: First 
Second Scale First Scale Conceptualization 
integration integration of the scale 
(24 items) (28 items) (30 items) 


Conceptualization 
Phase 


Pilot Survey E Main Survey | 


(24 items) (24 items) 


Reliability tests and 
Explorative Factor 
N Analysis (EFA) 


Confirmative 


Testing 
Phase 


Final version of scale Factor Analysis 
(CFA) 
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Analisis Buti Analisis 


Tes Klasik Faktor 


Step 3: Designing and Conducting Studies to Develop and Refine the Scale 
Issues to Consider: 
(a) Pilot testing as an item-trimming procedure 
(b) The use of several samples from relevant populations for scale 
development 
c) Designing the studies to test psychometric properties 
(d) Initial item analyses via exploratory factor analyses (EFAs) 


(e) Initial item analyses and internal consistency estimates 
nitial estimates of validity 
(g) Retaining items for the next set of studies 


Netemeyer, 2003 


Apa alasannya? 


Analisis butir perlu dilakukan 
per dimensi sehingga analisis 
taktor diperlukan di awal 


Stage 1 Scale 
Generation and 


Stage 2 Scale 
Purification through 


Exploratory Factor 


Stage 3 Reliability 
and Validity 


Stage 4 
Introduction EXQ 


Initial Purification 


- Using insight from 
literature review to 
choose context 
- In-depth interviews 
(n=30) 
- Generate initial pool 
of items 
-Readability Check 
(n=10) 
- Expert judgment 
tasks to assess face 
and construct validity 
(n=7) 
-Q-sorting (8 
categories) 
- Initial purification 
and categorization 
a)n=7 
b)n=5 
- Produced 37 items 
in 5 dimensions for 
next stage 


kol 


Qualitative Enquiry 
and Initial 
Purification 


Analysis (EFA) 


- Collect 
responses from 
representative 
sample (n=75) 

- Exploratory 
factor analysis (on 
5-dimensional 
model) 

- Assess content 
validity, scale 
reliability and 
validity 

- Develop purified 
scale with 19 
items in four 
dimensions for 
next stage 


Purification and 
Refinement 


Assessment 


- Collect responses 
from additional 
representative 
sample (n=218) 

- Confirmatory 
factor analysis (on 
4-dimensional 
model) 

- Assess factor 
structure and 
dimensionality of 
scale 

- Assess model fit 
-Assess scale and 
construct reliability 
and discriminant 
validity 

- Final EXQ scale of 
19 items in four 
dimensions 


Scale 


- Final EXQ scale 
of 19 items in four 
dimensions, 
conceptual model 


— 


Final Refinement 
and Validation 
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Netemeyer, 2003 


Apa alasannya? 


Analisis faktor memiliki fungsi 
sama dengan analisis butir namun 
lebih tepat karena sesuai dengan 
model 


Le aps go Assessment Copyright 1995 by the American Psychological Association, Inc. 
1995, Vol. 7, No. 3, 309-319 1040-3590/95/$3.00 


Constructing Validity: Basic Issues in Objective Scale Development 


Lee Anna Clark and David Watson 
The University of lowa 


A primary goal of scale development is to create a valid measure of an underlying construct. We 
discuss theoretical principles, practical issues, and pragmatic decisions to help developers maximize 
the construct validity of scales and subscales. First, it is essential to begin with a clear conceptualiza- 
tion of the target construct. Moreover, the content of the initial item pool should be overinclusive 
and item wording needs careful attention. Next, the item pool should be tested, along with variables 
that assess closely related constructs, on a heterogeneous sample representing the entire range of the 
target population. Finally, in selecting scale items, the goal is unidimensionality rather than internal 
consistency; this means that virtually all interitem correlations should be moderate in magnitude. 
Factor analysis can play a crucial role in ensuring the unidimensionality and discriminant validity 
of scales. 


Scale development remains a growth industry within psychol- signed to be completed by clinicians, parents, teachers, spouses, 
ogy. A PsycLIT database survey of articles published in the 6- peers, and so forth. 
year period from 1989 through 1994 revealed 1,726 articles Before proceeding further, it is interesting to examine the new 
with the key words “test construction” or “scale development” measures comprising our Psychological Assessment sample. This 
published in English-language journals, 270 in other-language examination sample offers a glimpse at why scale development 
journals, and 552 doctoral dissertations. During this same pe- continues unabated, as well as the nature of the unmet needs these 
riod (i.e., beginning with its inception), 50 articles addressing scale developers are seeking to fill. First, not surprisingly given this 
scale development or test construction were published in Psy- — journal's focus, more than half (61%) of the scales assess some 
chological Assessment alone. The majority of these articles re- aspect of psychopathology, personality, or adjustment. The next 
ported the development of one or more new measures (82%); most common categories are measures of attitudes and interper- 
most of the rest presented new scales derived from an existing sonal relations (20% and 15%, respectively). The remaining scales 
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PRI ersonality and Individual Differences 
FA P lity and Individual Diff 
e ke Volume 12, Issue 3, 1991, Pages 291-294 


Does item homogeneity indicate internal 
consistency or item redundancy in psychometric 
scales? 

Gregory ). Boyle 

Show more v 


+ Add to Mendeley « Share #5 Cite 


https://doi.org/10.1016/0191-886%91)90115-R Get rights and content 
Abstract 


The term ‘internal consistency’ has been used extensively in classical psychometrics 
to refer to the reliability of a scale based on the degree of within-scale item 
intercorrelation, as measured by say the split-half method, or more adequately by 
Cronbach's (1951) (Psychometrika, 16, 297-334) alpha, as well as the KR20 and KR21 
coefficients. This term is a misnomer, as a high estimate of internal item 
consistency/item homogeneity may also suggest a high level of item redundancy, 


wherein essentially the same item is rephrased in several different ways. 


Specifying the domain 


OSS projects hosted on Sourceforge 


Generating of initial scale items 


Literature review, Studying Sourceforge.net, Examining possble queries on enhancement 
process; Result=9 initial items 


Data Collection: first stage 


Collecting data on the 9 items from 240 OSS projects which adopted enhancement tracking 
system; Result=data collected on 9 items from 240 projects 


Scale Purification 
Assess convergent validity of items (CFA, Cronbach's alpha, AVE); Assess discriminant 


validity (CFA); Result= Convergent and discriminant validity were found for 6 out of 9 
measures; List of 6 candidate measures 


Data Collection: second stage 


Collecting data on the 6 items from 750 OSS projects to evaluate purified items; 
Result=data collected on the 6 items from 750 projects 


Assessing validity and reliability 


Assess composite reliabilities and AVE using PLS; Assess convergent and 
discriminant validity using PLS; Result = Convergent and discriminant validity were 
found for 4 of the purified measures; final list of 4 measures 
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Item Reliability Statistics 
if item dropped 
item-rest correlation Cronbach's a 
bi_01 0.7826 0.904 
bi_02 0.0501 0.919 Item Reliability Statistics 
if item dropped 


item-rest correlation Cronbach's a 


bi_02 0.678 0.87 

bi_04 0.792 0.857 
bi_07 0.685 0.869 
bi_09 0.797 0.857 
bi_10 0.525 0.888 
bi_12 0.429 0.891 
bi_13 0.598 0.877 
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e Scaling - Linking - Equating 


Analisis Faktor 


e Dipakai untuk mengidentifikasi struktur pengukuran 

e Struktur pengukuran menunjukkan berapa faktor atau dimensi dalam 
suatu konstruk ukur alat atau alat ukur serta keterkaitan di antara 
dimensi tersebut 

* Melalui analisis faktor peneliti mengembangkan model tertentu 
untuk memaksimalkan varians skor hasil pengukuran 


Total Variance 


Unique (residual) 


Unique 
Variance 


Sistematik Random 


1. Saya suka berteman [Ya] 
2. Saya suka menyapa orang lain [Ya] 
3, Saya suka menolong [Tidak] 


Total Variance 


Unique (residual) 


md 


Specific 


Unigue 
Variance 


Unigue 
(residual) 


Sistematik Random 


Ilustrasi 


Item Reliability Statistics 
if item dropped 


item-rest correlation Cronbach's a 


bi 01 0.7826 0.904 
| 
bi 03 0.7762 0.905 Total Variance 
F 0.918 
0 
0 


Unique (residual) 


Item Reliability Statistics 
If item dropped 


item-rest correlation Cronbach's a 


bi 02 0.678 0.870 Total Variance 


ki N A 707 0 257 
bi 04 0.792 0.85 
hi 07 ROG N 26 
bi 07 0.685 0.869 


bi 09 0.797 0.857 Conn = Unique 
i Specific E 


bi 10 0.525 0.888 mon (residual) 
bi_12 0.429 0.891 
bi_13 0.598 0.877 
bi_17 0.826 0.855 


Kapan Menggunakan Analisis 
Faktor? 


e Mengidentifikasi struktur data 
e Menguji validitas faktorial/konstruk/struktural 
e Menguji validitas divergen/konvergen 


Semoga Bermanfaat 


